1. Field of Invention
The present invention relates to visual processing devices, in particular to visual processing devices that perform visual processing such as spatial processing or gradation processing of an image signal. Separate aspects of the invention relate to visual processing methods, visual processing programs, integrated circuits, display devices, image-capturing devices, and portable information terminals.
2. Description of the Related Art
Spatial processing and gradation processing are known as techniques for visually processing image signals of an original image. Spatial processing involves using the pixels around a pixel to be filtered when processing that pixel. Further, the technique of using an image signal that has been subjected to spatial processing to perform contrast enhancement or dynamic range (DR) compression, for example, of an original image is known. With contrast enhancement, the difference between the original image and the blur signal (the sharp component of the image) is added to the original image, sharpening the image. With DR compression, a portion of the blur signal is subtracted from the original image, compressing the dynamic range.
Gradation processing is processing in which a lookup table (LUT) is used to transform a pixel value for each pixel being processed without regard for the pixels around the pixel being processed, and is also referred to as “gamma correction.” For example, in the case of contrast enhancement, transformation of the pixel value is performed using a LUT that produces a gradation of gray levels that appear frequently (whose area is large) in the original image. Well-known examples of gradation processing using a LUT include gradation processing in which a single LUT is chosen and used for the entire original image (histogram equalization) and gradation processing in which the original image is partitioned into a plurality of image regions and a LUT is chosen and used for each image region (local histogram equalization) (for example, see JP 2000-57335A (pg. 3, FIGS. 13 to 16)).
An example of gradation processing in which an original image is partitioned into a plurality of image regions and a LUT is chosen and used for each image region is described using FIGS. 104 to 107.
FIG. 104 shows a visual processing device 300 that partitions an original image into a plurality of image regions and chooses a LUT to use for each image region. The visual processing device 300 is provided with an image partitioning portion 301 that partitions an original image that has been input as an input signal IS into a plurality of image regions Sm (1≦m≦n; where n is the number of partitions of the original image), a gradation transformation curve derivation portion 310 that derives a gradation transformation curve Cm for each image region Sm, and a gradation processing portion 304 that obtains the gradation transformation curves Cm and subjects each image region Sm to gradation processing and outputs the result as an output signal OS. The gradation transformation curve derivation portion 310 comprises a histogram creation portion 302 that creates a brightness histogram Hm for each image region Sm, and a gradation curve creation portion 303 that creates a gradation transformation curve Cm for each image region Sm from the brightness histogram Hm that has been created.
The operations of these portions are described using FIGS. 105 to 107. The image partitioning portion 301 partitions an original image that has been received as an input signal IS into a plurality (n) of image regions (see FIG. 105(a)). The histogram creation portion 302 creates a brightness histogram Hm for each image region Sm (see FIG. 106). Each brightness histogram Hm shows the distribution of the brightness values of all pixels in an image region Sm. That is, the horizontal axes in the brightness histograms Hm shown in FIG. 106(a) to (d) show the brightness level of the input signal IS and the vertical axes show the pixel number. The gradation curve creation portion 303 cumulates the “pixel number” of the brightness histogram Hm in the order of their brightness and this cumulative curve is taken as a gradation transformation curve Cm (see FIG. 107). In the gradation transformation curve Cm shown in FIG. 107, the horizontal axis shows the brightness value of the pixels of the image region Sm in the input signal IS, and the vertical axis shows the brightness value of the pixels of the image region Sm in the output signal OS. The gradation processing portion 304 obtains the gradation transformation curve Cm and transforms the brightness value of the pixels in the image region Sm in the input signal IS based on the gradation transformation curve Cm. By doing this, a gradient is established between the most frequent gradations in each block, and this increases the sense of contrast for each block.
Visual processing that combines spatial processing and gradation processing also is known. Conventional visual processing that combines spatial processing and gradation processing is described below using FIG. 108 and FIG. 109.
FIG. 108 shows a visual processing device 400 that performs edge enhancement and contrast enhancement utilizing unsharp masking. The visual processing device 400 shown in FIG. 108 is provided with a spatial processing portion 401 that performs spatial processing with respect to the input signal IS and outputs an unsharp signal US, a subtracting portion 402 that subtracts the unsharp signal US from the input signal IS and outputs a difference signal DS, an enhancing portion 403 that performs enhancement of the difference signal DS and outputs an enhanced signal TS, and a summing portion 404 that takes the sum of the input signal IS and the enhanced signal TS and outputs an output signal OS.
Here, enhancement processing is performed with respect to the difference signal DS using a linear or non-linear enhancement function. FIG. 109 shows the enhancement functions R1 to R3. The horizontal axis in FIG. 109 marks the difference signal DS and the vertical axis marks the enhanced signal TS. The enhancement function R1 is an enhancement function that is linear with respect to the difference signal DS. The enhancement function R1 is a gain adjustment function expressed for example by R1(x)=0.5x (where x is the value of the difference signal DS). The enhancement function R2 is a non-linear enhancement function with respect to the difference signal DS, and is a function that inhibits extreme contrasts. In other words, a greater inhibitory effect (an inhibitory effect due to a larger inhibition rate) is exhibited with respect to an input x having a large absolute value (where x is the value of the difference signal DS). For example, the enhancement function R2 is expressed by a graph having a smaller slope the larger the absolute value of the input x. The enhancement function R3 is a non-linear enhancement function for the difference signal DS that inhibits the noise component in small amplitudes. That is, a greater inhibitory effect (an inhibitory effect due to a larger inhibition rate) is attained with respect to an input x having a small absolute value (where x is the value of the difference signal DS). For example, the enhancement function R3 is expressed by a graph having a larger slope the greater the absolute value of the input x. The enhancing portion 403 can use any of these enhancement functions R1 to R3.
The difference signal DS is the sharp component of the input signal IS. With the visual processing device 400, the intensity of the difference signal DS is transformed and added to the input signal IS. Thus, the output signal OS is the input signal IS in which the edges and the contrast have been enhanced.
FIG. 110 shows a visual processing device 406 that improves the local contrast (intensity) (for example, see Japanese Patent JP 2832954 (pg. 2, FIG. 5)). The visual processing device 406 shown in FIG. 110 is provided with a spatial processing portion 407, a subtracting portion 408, a first transformation portion 409, a multiplying portion 410, a second transformation portion 411, and a summing portion 412. The spatial processing portion 407 performs spatial processing with respect to the input signal IS and outputs an unsharp signal US. The subtracting portion 408 subtracts the unsharp signal US from the input signal IS and outputs a difference signal DS. The first transformation portion 409 outputs an amplification coefficient signal GS for locally amplifying the difference signal DS based on the intensity of the unsharp signal US. The multiplying portion 410 takes the product of the difference signal DS and the amplification coefficient signal GS and outputs a contrast enhanced signal HS in which the difference signal DS has been locally amplified. The second transformation portion 411 locally corrects the intensity of the unsharp signal US and outputs a corrected unsharp signal AS. The summing portion 412 takes the sum of the contrast enhanced signal HS and the corrected unsharp signal AS and outputs an output signal OS.
The amplification coefficient signal GS is a non-linear weight coefficient that locally corrects the contrast in portions of the input signal IS where the contrast is unsuitable. For this reason, portions of the input signal IS having suitable contrast are output unchanged, and those portions with an unsuitable contrast are corrected and then output.
FIG. 111 shows a visual processing device 416 that performs compression of the dynamic range (for example, see JP 2001-298619A (pg. 3, FIG. 9)). The visual processing device 416 shown in FIG. 111 is provided with a spatial processing portion 417 that performs spatial processing with respect to the input signal IS and outputs an unsharp signal US, a LUT computation portion 418 that uses a LUT to perform an inverse transformation of the unsharp signal US to produce a LUT processed signal LS which it then outputs, and a summing portion 419 that takes the sum of the input signal IS and the LUT processed signal LS and outputs an output signal OS.
The LUT processed signal LS is added to the input signal IS to compress the dynamic range of low-frequency components of the input signal IS (frequency components lower than the cutoff frequency of the spatial processing portion 417). As a result, the dynamic range of the input signal IS is compressed while its high-frequency components are retained.